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1. Introduction 

A combinatorial optimization problem is defined, in a broad sense, by specifying a 
certain number of free variables constituting it and the conditions that its solution must 
satisfy. In treating an optimization problem the fundamental step is to find the most 
efficient algorithm yielding the solution. Efficient from the point of view of all the 
computing resources needed for its performance, the most important of which is the 
time requirement, and, in particular, its dependence on the size of the problem. By 
size we mean, in an informal way, the number of variables, or, even, the number of 
conditions. 

A computational hard problem 0, 0] is an optimization problem for which the time 
needed to find the solution, or to determine with certainty that it has no solution, very 
sensitively increases with the size. More specifically there are no polynomial algorithms 
able to solve it. This kind of problems are therefore called intractable or Non Polynomial 
(NP). 

Any NP problem can be reduced to a particular NP paradigmatic problem (the SAT 
problem) exploiting algorithms performing such a mapping in a polynomial time 0, ^. 
Computer scientists call this class of very hard combinatorial optimization problems the 
class of Non Polynomial- Complete (NPC) problems 

Summing-up, a NPC problem is defined, in a qualitative way, as an optimization 
problem whose solutions, or the certainty that it has no solution, can only be found, in 
the worst case, by algorithms whose computation time grows faster than any polynomial 
with the number of variables of the system. 

In this paper we shall consider the 3-SAT problem 0]. This is a particular version 
of SAT problem, which is the paradigm of NPC combinatorial problems showing a phase 
transition. For this problem the free variables are boolean variables and the conditions 
are sets of three boolean numbers. Its importance, apart from being a historical one, 
comes from the fact that even if theoretically any NPC problem can be mapped in any 
other, in practice given problems are better suited for proving such correspondence. Of 
these reference problems one of the most useful (and used) is the 3-SAT. 

In the last years a one to one correspondence has been observed between 
computational hard problems and the ground state properties of spin-glass models 
0, 1^. Statistical mechanics has been applied to the study of universal behaviour in 
the computational cost of some class of algorithms, searching for solutions of random 
realizations of the prototype of the NPC problems: the Satisfaction (SAT) problem 
10]. The investigation of the properties of NPC problems is then performed through the 
introduction of an energy or cost function and an artificial temperature. In this mapping 
the actual NPC problem is recovered as the T = limit of the associated statistical 
mechanical problem. Such an approach has been implemented both numerically, using 
simulated annealing algorithms P]), and analytically. 

To set up a statistical mechanical approach one first introduces a semidefinite 
positive Hamiltonian Ti.[C] function, defined for each given instance C of the problem. 
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constructed in such a way that if the configuration C* is solution of the computational 
problem then 7i[C*] = 0. On the contrary, if Ti.[C] > for any C then problem does not 
admit solution. Having defined an Hamiltonian, the associated statistical mechanical 
problem is described by the partition function 

Z(/5)=^exp(-/3H[C]) , (1) 

c 

where f3~^ = T is the (artificial) temperature of the system. Proceeding further one 
introduces the usual thermodynamic quantities, e.g., the energy 

UiP, . -M) . (2) 

The mapping is not trivial since intensive quantities, such as the energy density 
u = U/N, do not depend on N in the infinite limit, so that a computation of their 
average over the distribution of instances of the computational problem is sufficient to 
obtain relevant informations on its satisfiability. 

Eventually, to recover the original computational problem, the limit T — > has to 
be taken. We stress that, in this approach, the temperature only plays a role for the 
constsruction of a statistical mechanics problem, of which the only interesting features 
are those at T = 0. 

In this approach, phase transition concepts play an important role to build a 
theory for typical case complexity in theoretical computational science. The importance 
comes from the fact that NPC decision problems, that are computationally hard in 
the worst case, may be not in the typical case, if one excludes the critical regions 
of the parameter space, where almost all instances become computationally hard to 
solve. The computational critical region corresponds to a phase transition region in 
statistical mechanics language. Far from phase boundaries the problems are either 
under-constrained or over-constrained and one can determine search procedures able to 
find solutions, or certainty of no solution, in polynomial times: the results of worst-case 
complexity theory are not very relevant in practice and what is necessary is a theory 
for typical-case complexity. To such a purpose, the analysis of general search methods 
applied to different classes of hard computational problems, characterized by a large 
number or relevant, randomly generated, variables, is fundamental. 

Variables are under-constrained when the minimal number of violated clauses does 
not depend on their possible assignments. In particular this is true when they do not 
appear in any clause. In the under-constrained phase the clauses of the problem can 
always be satisfied (SAT phase). On the contrary, variables are over-constrained when 
they cannot satisfy simultaneously all the clauses imposed on them. In this case we 
are in the UNSAT phase. Going back to the mapping onto a statistical mechanical 
problem, the UNSAT-phase corresponds to a frozen (spin-glass) phase while the SAT- 
phase corresponds to an ordered (ferromagnetic) phase. 

The 3-SAT problem, and, in general, the ii'-SAT problem where the clauses contain 
a number K of elements, can be mapped onto a diluted long-range spin-glass model 
P]. The model is mean-field because of the lack of geometrical correlations in the 
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clauses. However, since each spin has only a finite number of neighbours strong local 
field fiuctuations, stronger than in those of fully connected spin-glass, are present. 

The relevant parameter driving the SAT/UNSAT transition is the ratio a between 
clauses and number of variables of the system, which is the connectivity in the statistical 
mechanical analogue of the combinatorial problem. 

Indeed too many conditions cause the unsatisfiability of the problem. The entropy 
of the associated spin model gives a measure of the typical number of solutions. 
Therefore at the transition an abrupt disappearance of all (exponentially numerous) 
solutions makes the entropy jump to zero. 

For K = 1 and 2 the problem is solvable: the time to find the solutions grows 
polynomially (actually even linearly 0) with the number of variables. For i^' > 3 
the problem is, on the contrary, NPC The transition threshold for 3-SAT has been 
determined numerically at a^S) ~ 4.25 — 4.30 0. 

Based on the mapping of random clauses onto the quenched disorder of the 
associated spin model, in Ref. || the Replica trick was introduced to compute the 
statistical mechanics of the K-SAT problem and the Replica Symmetric (RS) theory was 
carried out. The K-SAT problem is naturally mapped onto a disordered spin model with 
finite connectivity, where the role of connectivity is played by the density of clauses. 
Even if it gives a quahtative good pattern of the transition it is, however, unable to 
predict correctly the value of the transition threshold between the SAT-phase and the 
UNSAT-phase and the correct thermodynamic quantities in the UNSAT-phase. 

The failure of the RS solution can be traced back to the existence of a very large 
number of equilibrium states of the associated statistical mechanical problem in the 
thermodynamic limit — > oo. To deal with those, and improve the knowledge of the 
structure of the solutions of the decision problem, it is necessary to break the Replica 
Symmetry. Replica Symmetry Breaking (RSB) in diluted models is a very hard issue, 
due to the complex structure of the saddle point equations, for recent approaches see 



la, 11 



As shown in [|T2[ the SAT/UNSAT transition results from the sudden freezing of a 
finite number of variables, as a increases above Oc- These variables form a backbone that 
does not disappear in the thermodynamic limit. Information about the structure of this 
backbone and about the mutual overlap between different assignments minimizing the 
cost function in the UNSAT phase is, then, very important to understand the transition. 

In an attempt to overcome the difficulty of solving a spin glass diluted model a 
variational approach, both for RS and 1-RSB solution, has been recently proposed. It 
is based on the existence of the backbone of over-constrained variable that remains 



finite in the thermodynamic limit |T3[. This circumvent the necessity of solving the 
self-consistent equations for RS and 1-RSB, but does not resolve the question about 
the nature of the RSB solution in the UNSAT-phase. Thus the relation between RSB 
transition and the typical case complexity theory is yet an open question. 

In order to investigate the the nature of the RSB solution in the UNSAT-phase, in 
this paper we shall consider the K-SAT problem with K = 3, the simplest NPC problem 
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of this class, in the hmit of a large number of clauses a ^ 1 (over-constrained), where 
the associated statistical mechanical problem can be handled with known techniques of 
disordered spin systems. The basic idea is that since there should be no other transitions 
for a > ac, the structure of the UNSAT-phase for a S> Oc should be representative 
of the whole UNSAT-phase in the range a > ac- Performing a careful study of the 
thermodynamic quantities down to zero temperature of the associated disordered spin 
model we find that the RSB is infinitely broken (oo-RSB) in the UNSAT-phase. 

Moreover, using the first two terms in the asymptotic expansion in l/i/a of the 
thermodynamic quantities, we can obtain an upper bound for a^- 

The paper is organized as follows. In Section | we introduce the model. The details 
of the derivation of the cxo-RSB solution with an arbitrary gauge are given in section 
The calculation is carried out for a general model introduced in Section |^, to which 
the over-constrained 3-SAT problem belongs. Here we also sketch the procedure used 
for the numerical of solution of the oo-RSB solution. The oo-RSB solution for the 
over-constrained 3-SAT model is discussed in Sections |] and |^. 



2. The over-constrained 3-SAT model 



The model we study has been introduced in where the analysis of RSB at one 
and two steps has been carried out. The 3-SAT model is defined by a set of boolean 
variables s{i) = 0, 1, defined on the sites i = 1, . . . ,N, and an ensemble of randomly 
generated 3-SAT boolean formulae. First the random boolean formulae are constructed 
by assign to each triplet {ii, 12, "is}, with ii < 12 < is, a set of three independent variables 
^1,2,3 which take the value +1 or —1 with probability 1/2. Next for each instance of 
the problem triplets of randomly chosen sites {ii, i2, is} are selected by assigning to the 
variables Ti^^i^^i^ the value 1 with probability p = aN~'^ and with probability 1—p. For 
N —>■ 00 there are aN variables r which are different from zero, and hence aN 3-SAT 
boolean formulae.. 

If we introduce the spin variables a{i) = 1 — 2s{i), the cost function reads 

1 - 6^'"'"V(.0 1 - er'"''-V(^,) 1 - 6g-"''-V(z3) . . 

n<«2<j3 

which is nothing but the number of unsatisfied clauses. Indeed it is easy to see that 
each term is either 1 (unsatisfied) or (satisfied). Note that 7i = if and only if all the 
clauses are satisfied. 

The statistical mechanical approach to the 3-SAT problem takes Ti. as the 
Hamiltonian of a disordered spin system, and, as discussed above, studies the properties 
of the ground state. 

The fundamental quantities in studying the hard optimization problems with the 
tools of statistical mechanics are the zero temperature energy and entropy densities, 
respectively uq and sq, in the thermodynamic limit, uq represents the average over the 
distribution of clauses of the number of clauses that are not satisfied by the formula 
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(0). So is the logarithm of the number of solutions satisfying the formula divided by 
the number of variables. For the behaviour of these quantities as functions of the 
connectivity-like parameter a, the conjecture is done0, ^ that 

Uo{a) = , So{a) > 0, for a < ac (4) 

Uo{a) > , So(a) = 0, for a > (5) 

For a <^ Oc the problem is quite under-constrained and it is relatively easy to find 
an assignment of variables cxj satisfying the clauses. In other words for a < ac the 
problem is SAT, with probability going to 1 for — > oo, On the contrary for a > 
the problem does not have solutions, UNSAT-phase. The analysis of the UNSAT-phase 
is in general rather hard. The most difficult case occurs around ac where an exponential 
time may be needed to determine the unsatisfiability. Away from the critical region, 
i.e., a S> ac, to prove unsatisfiability is easier, and more insight into the structure of 
the phase space can be gained. 

In the present paper we will work in the over-constrained approximation a ^ ac, 
where the computation strongly simplifies. This limit is obtained by expanding in 1/ ^/a 
to second order, after having rescaled the temperature /? — >• /?/ y/a . For further details 



See 



Note that in Ref. |T^ the reduced inverse temperature was (3 = [ij \fa. Here we 



will, instead, keep the notation (5 also for the reduced inverse temperature. Moreover in 



the clauses are erroneously over counted in the evaluation of the partition function 
I5| . This, however, does not produce any relevant change, apart from a rescaling of the 
reduced temperature and of the energy and the free energy of a factor 1/a/6, leaving 
the entropy invariant.|J] In order to make a comparison with the results shown there, it 
is enough to multiply /3, the free energy and the energy shown in the present paper by 
a factor y^. 

3. The replica approach in a generalized form 

The 3-SAT model belongs to the family of spin models interacting via quenched random 
couplings. These are described by a random Hamiltonian H[J;o"] where J are the 
random "quenched" couplings. For example, in the Sherrington- Kirkpatrick (SK) model 
J is a symmetric Gaussian matrix of zero mean and variance proportional to 1/A^ ||16|| , 
while in its p-spin generalization the variance goes like For the 3-SAT 

problem the disorder is introduced by the random clauses imposed on the set of variables. 
In the simple limit that we are considering here, the quenched disorder is represented 
by the random ±1 variables e(*i'*2.«3)^ assigning a clause on the three sites ii, 12 and 23 

0- 

For any fixed coupling realization J, the partition function of the spin system, with 

f In [p^ the reduced inverse temperature was called /i, therefore the substitution /i = \/6/3 cure the 
difference. 
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N spins, is given by [|T8|, [19 1 

Zjv[J] =Tr. exp{-(3n[J;a]) 
and the quenched free energy per spin is 



Np 



N 



Np 



d[J] P[J] lnZjv[J] 



(6) 



(7) 



where (■ ■ ■) indicates the average over the couphngs reahzations. We assume that the 
thermodynamic hmit of the free energy, — hmAr^oo ln2'Af[J] / NP is well defined and is 
equal to the quenched free energy / = lim^r^oo In for almost all coupling realizations J 
(self-average property). 

The analytic computation of the quenched free energy, i.e., of the average of the 
logarithm of the partition function, is a quite difficult problem, even in simple cases as 
nearest neighbour one dimensional models. However, since the integer moments of the 
partition function are easier to compute, the standard method uses the so called "replica 
trick" by considering the annealed free energy f{n) of n non- interacting 'replicas' of the 



system M, 18, 19 



fin) 



lim 



In 



{ZnIJ])"" 



(8) 



N^oo NPn 

The quenched free energy of the original system is then recovered as the continuation 
of f{n) down to the unphysical limit n = 0, 



f 



lim lim 



Npn 



lini/(n). 

n-+0 



(9) 



In the last equality we assumed that the replica limit and the thermodynamic limit 
can be exchanged. This procedure replaces the original interactions in the real space 
with couplings among different replicas. The interested reader can find a complete and 
detailed presentation of the replica method for disordered statistical mechanical systems 
in Ref . [|l|] and in Ref . pj . 



In what follows we shall consider disordered spin systems for which / in the replica 
space can be written in the form 



Pf [Qab,Aal]=PUP) 



1 



1^2 I l,n 

P' 



l,n 



1 



lini - log Tr^ exp — ^ k^h a"'o- 



(10) 



where Qab is the spin-overlap matrix in the replica space between replicas a and h: 



N 



Q 



ab 



i=i 



and Aab, the Lagrange multiplier associated with Qab, gives the interaction matrix 
between spins of different replicas. Angular brackets denote thermal average. 
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Stationarity of / with respect to variations of Aab and Qab leads to the self-consistency 
equations for the matrices A and Q: 

Aab = 91 {Qab) (12) 

Tr, aV''exp(f Ea^bAai^V 



Tr,^ exp Y.aj^b Kbcr^cr^ 
where we have used the short-hand notation 



9n[Z) 



n 



1,2,... 



(13) 



(14) 



the SK model we have[16 



The function g and the constant /o depend on the specific model. For example for 

(15) 



9[Z) = y, 



fo 



4' 



Similarly the p-spin model is recovered for 



9{z) = y, 



fo 



"4 



Finally, for the 3-SAT problem in the limit of low dilution we have || [14]: 

1 



9[z) 



384 



1 + z) 



f = - A + 

96 48 



(16) 



:i7) 



In the following we will use /o = —(3/96, the only consequence being a shifting of the 
free energy density / and of the internal energy density -u by a factor y/a/AS. 



4. Infinite Replica Symmetry Breaking Solution 

oo-RSB solution 



To evaluate the n — > limits in (|T0|) one has to make an Ansatz on the structure 
of matrices A and Q, i.e., to choose a Replica Symmetry Breaking (RSB) scheme. 
In order to be as general as possible, we shall use the RSB scheme introduced by 
de Dominicis, Gabay and Orland ||20|, which besides the Edwards- Anderson order 
parameter also involves the anomaly to the linear response function, also called 
Sompolinsky's anomalies The more usual Parisi's RSB scheme is recovered by a 
proper gauge fixing. Here we shall only report the main results, since the calculation is 
straightforward. The interested reader can find some details in Ref . |2T| . 

By applying the RSB scheme infinite times and introducing two functions q{x) and 
X{x), < a; < 1, one for each matrix, the free energy functional (0) becomes [^, [2l[] : 

/?/(/?) = (3fo{(3) + ^[9 (g(l)) + A(l) (1 - g(l))] +^ f'dx 9, {q{x)) A,{x) 
I z Jo 



"2 



dx 



q{x)Ax{x) + X{x)Ag{x) -(3 



+ 00 



dy 



: exp 



2A(0) 



(18) 

my) 



/27rA(0) 

X In it was /o = — ^ + ^ and g{z) = ^(1 + zY. Due to the over-counting discussed at the end 
of the introduction (i.e. /i/ = 6(3 f), these were six times bigger than the actual definition. 



The 3- SAT problem with large number of clauses in the oo-RSB scheme 



9 



where 0(0, y) is the solution evaluated at x = of the the Parisi's equation 

y) = y) + ^ yY (19) 

with the boundary condition 

0(l,t/) = Tlog(2cosh/?y) (20) 

and Aq(x) and Ax{x) are the anomalies associated with the order parameters q{x) 
and X{x). We have used the standard notation and denote derivatives with respect 
to a; by a dot and derivatives with respect to ?/ by a prime. Note that with this 
notation Sompolinsky's A' becomes TA. It is easy to see that using (|15D one recovers 
the Sompolinsky functional for the SK model pSf, and inserting the Parisi's gauge 
Ag(x) = —l3xq{x) the Parisi's functional [24]. 

The Parisi's equation (|19D can be included into the free energy via the Lagrange 
multiplier P{x,y) and the initial condition at x = 1 (pO|) via P{l,y). The free energy 
then becomes ll2J 



r' + OO 

f3W)=f3f{P)+f3 I dy P{l,y) [0(1, y) - Tlog (2 cosh/^y)] (21) 



oo 



PL f + OO 

- 13 dx dy P{x, y) 

Jo J-oo 



{x,y) + ^<P"{x,y)-^{<P\x,y)f 



By this construction is stationary with respect to variations of P{x,y), P{l,y), 
(f){x,y), (j){0,y), the order parameters q{x) and X{x) and anomalies Ag(x) and Ax{x). 
Variations with respect to P{x,y) and P{l,y) simply give back eqs. (|19[) and (|20|). 
Stationarity with respect to variations of 0(x, y) and 0(0, y) leads to a partial differential 
equation for P{x, y): 

P{x, y) = ^ P"{x, y) + A,(x) [P(x, y) 0'(x, y)l (22) 
with the boundary condition at x = 

my)=r^exp(--4). (23) 

J-^ V27rA(0) V 2A(0); 

Finally, variations of g(x), Ag(x), A(x) and Aa(x) lead to 

Ax{x)=g2{q{x))A,{x) (24) 

\{x) = g, (g(x)) (25) 
A,(x) = -/3[l- g(l)] + / dy P(x, y) 0"(x, y) (26) 



qix)= dyP{x,y)ct)'{x,yf (27) 



with Ag(l) = 0, the anomalies at the shortest time-scale, corresponding to x = 1, being 
zero by construction. 
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The Lagrange multipher P{x, y) gives the probabihty distribution of local fields. 
One may indeed associate a given overlap q{x) with a time scale r^. such that for times 
of order Tx states with an overlap equal to q{x) or greater can be reached by the system. 
In this picture the P{x, y) becomes the probability distribution of frozen local fields y 



at the time scale labeled by a; |25 



By partial derivate the above expressions we can obtain some useful relations. For 
example, deriving with respect to x equations (|27|) and (^Sf) , or equivalently (pBf ) and 
(1^), one gets 

g2{q{x)) / dyP{x,y)<P"{x,yf = l. (28) 



A further derivation with respect to x leads to 

/oo 
dyP{x,y) [\{x) <P"\x,yf + 2Aa(x) <P'\x,yf] = 0. (29) 
-oo 

Using equations ( pSf ) and (p^) this becomes 

q{x) \{x) 2jdy??^P{x,y)r{x,yy 



A,(x) Aa(x) g^iqix)) + g2{q{x)) JZodyP{x,y) 0'"(x,i/) 



2 ■ 



(30) 



which determines the gauge relation between q{x) and Ag(a;) and between A(x) and 
Aa(x), the Parisi's Px. 

Finally we note that from eq. ( p7| ) and the interpretation of P{x,y) distribution 
of local fields, m{x,y) = (j)'{x,y) can be interpreted as the local magnetization over the 
time-scale x. It obeys the equation 

\(x) 

m(x, y) = — m"{x, y) + Aa(x) m{x, y) m'{x, y) , (31) 

with initial condition 

m(l,y) =tanh(/3y). (32) 

In the next Section we shall report the results of numerical integration of the above 
equations for the specific case of 3-SAT in the limit of a large number of clauses. 

4-2. Thermodynamic quantities 

Since the free energy density [eq. (pT])] is stationary we can easily calculate 
thermodynamic derivatives to compute for example the the energy density u: 

« = ^ = ^ /?/o + P[9 (g(l)) + A(l) (1 - g(l))] -\[dx q{x) A,{x) (33) 

/OO nOO nOO 

dy P(l,y) 0(1,1/) - / dy PiO,y) 0(0, y) - / dy P(l,y) y tanh(/?y). 
-OO J —OO J —oo 

This expression can be simplified using the relation 



d2/P(l,y)0(l,y)- / rfyP(O,y)0(O,y) = -- / dxqix)Axix). (34) 

D 2 Jo 



which follows computing Jq dx f^^dyP{x,y) (j){x,y) using either ( |T9|) or (|2^) and 
equating the results. 
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We can equivalently compute u by taking the derivative of the free energy density 
(p!0|) as a function of the generic matrix Qab, before any RSB scheme is introduced: 

d d 1 

^ /5/ [Qab, K,] = MP) + Pq^UP) - - (Qab) (35) 

By inserting now the chosen RSB scheme, and taking the — > hmit, we obtain the 
alternative form 

u = UP) + P^foiP) + Pg + £ dxg^ (g(x)) A,(x). (36) 

Note that by equating (^) and (|36|) we get another integral relation: 
/ dx[q{x) Aa(x) + A(x)A,(a;)] = - / dy P(l, y) y tanh(/5y) +P[l- q{l)]. (37) 

Jo J —oo 

Similarly we easily obtain the entropy density, 

s = + y19 m) + m (1 - im m 

/oo 
dy P{l,y)[\og2 cosh Py-y tanh{py)]. 
-oo 

4-3. Numerical Integration of the oo-RSB equations 

In order to study the low temperature regime of the 3-SAT in the limit of a large 
number of clauses we have numerically integrated the cxd-RSB equations to determine 
q{x), P{x, y) and m(x, y). We followed the iterative scheme of Ref. but with an 

improved numerical method which allows for very accurate results for all temperatures. 

We start from an initial guess for q{x), then m{x,y), P{x,y) and the associated 
q{x) are computed in the order as: 



(i) Compute m{x, y) integrating from x = 1 to a; = eqs. (0) with initial condition 

(H). 

(ii) Compute P{x, y) integrating from a; = to x = 1 eqs. ( P^ ) with initial condition 

(iii) Compute q{x) using eq. (|27|). 

The steps 1. ^ 2. ^ 3. are repeated until a reasonable convergence is reached, typically 
mean square error on q, P and m is of the order O(10~^). 

The core of the integration scheme is the integration of the partial differential 
equations (|3TD and (E^). In previous works this was carried out through direct 



integration in the real space which requires a large grid mesh to obtain precise results. To 



overcome such problems we use a pseudo-spectral dealiased ||2^ code on a grid mesh 
of X Ny points, which covers the x-interval [0,1] and the y-interval [— y max, Umax]- 
De-aliasing has been obtained by a N/2 truncation, which ensure better isotropy of 
numerical treatment. The x integration has been performed using an third-order Adam- 
Bashfort scheme. Typical values used are N^ = 100 -h 5000, Ny = 512 4096 and 
Umax = 12 -h 48. The number of iterations necessary to reach a mean square error on q, 
P and m of order O(10~^) is few hundreds. More details can be found in Ref. |2^ . 
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5. oo-RSB solution of the Highly Constrained 3-SAT problem 

In the numerical solution of the cxd-RSB equations we used different gauges depending on 
the temperature range. The reasons is that the Parisi's gauge = —^xq (A^ = —/SxX), 
which uses a simple relation between order parameters and anomalies, leads to numerical 
instabilities for large f3 since it is coupled with a (numerical) derivative. On the contrary, 
since in this gauge the derivatives goes to zero as x — 1, it is rather useful for not too 
large values of (3. Typically for T larger than 0.02 0.04. 

The overlap q{x) for different temperatures is shown in Figure (|I]). The transition 
between T = 0.0817 and T = 0.0898 is easily recognizable from the deviation of 
q{x) from a constant (the critical value at which the RS solution breaks down is 
Tc = 0.089725). The rounding near the plateaus is an artifact of finite A^^;. Indeed 
for increasing the shoulder becomes steeper and steeper and, in the limit A^^; — * oo, 
q{x) develops a discontinuity at the end points of the plateaus |2^. By varying the 
extrema of the x-integration and A^^^ the plateaus end-points xi and X2 can be precisely 
identified. On then concludes that the functional form of q{x) is similar to the one of 
the SK model in external magnetic field: 

{g(0) < X < xi 

non trivial Xi < x < X2 (39) 
g(l) X2 < X < 1 

The analytic form of the non trivial part of q{x) could be obtained from the resummation 
of high order expansions of the oo-RSB equations similarly to what is done for the SK 
model However, since observables such as q{0), q{l), energy, etc., are not very 

sensitive (difference of the order of the numerical precision) to the smoothness of q{x) 
we did not performed such an analysis here. 

In Figure (^ the behaviour of the largest and smallest overlaps g(l) and g(0) as 
function of temperature is compared with the results from one and two RSB solutions. 

For lower temperatures we used the Sommers' gauge which takes an anomaly with 
constant derivative At difference with the SK, here we have two anomalies and 

hence two possible choices. However, the more natural one for numerical integration 
Ax{x) = —Ax{0) = const leads to a more involved determination of Aa(0). Indeed 
we should first find Ag(x) from eq. ( ^6]) and then Aa(0) from (p4|). Therefore for low 
temperatures we adopted the Sommers' gauge Aq(x) = — Ag(0) = const , where [see eq. 

my- 

/oo 
dyPiO,y)m'iO,y) (40) 
-00 

and Aa(x) = —Ag{0)g2{q{x)). We note that since Aa does not vanish for x — > 1 this 
leads to numerical instabilities for large temperatures. Therefore from the point of view 
of numerical integration the two gauges are complementary. 

The order parameters g(x) and A(x) are different if we use the Parisi's or the 
Sommers' gauge, but the thermodynamics observables are, of course, invariant. This 
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Figure 2. The order parameter at the slowest {x — 0) and at the fastest {x — 1) 
time scales as a function of reduced temperature. As the RSB scheme is improved the 
splitting between the two values increases. 



fact has been used to check the numerical integration by comparing the results from the 
two gauges in the temperature range where both are stable. 

One of the main advantage of Sommers' gauge is that we can solve the equations 
at exactly T = 0. In Figure (0), for example, we report q{x) for T = in the Sommers' 
gauge. We recall that in the Parisi's gauge q{x) = q{l) for x > but g(0) 7^ ^(l), as 
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q(x) 




Figure 3. The order parameter q{x) at zero temperature for the 3-SAT, in the gauge 



-A,(0). 



For what concerns the thermodynamic quantities one sees that using (^) and (p^) 
the entropy must be proportional to for T — > and hence vanishes. Moreover it also 
follows that in the same hmit g(l) ~ 1 — aT^ |SD| . 

It can be easily checked that 



lim 

T->0 



lim 



mo 

dp 







so that the energy density for T = can be written as 



(41) 



(42) 



The last term can be expressed as function of the local magnetization using the identity 



where 



0(0, y) = 0(0,0)+ / dy^miO,y, 



0(0,0) 



(43) 



1 /■! 

dy{l-m{0,y))-- dx Ax{x) 
2, Jo 

1 

dy{l-m{0,y)) + -Ag{0) dxg2{q{x)) 

2 Jo 



(44) 

Using the relations derived in the previous section, alternative expressions for u 
can be obtained. For example by means of (0) evaluated for T = the energy density 
takes the form 



f = u 



dx q{x) Ax{x) - 2 dy y P{l,y) 
Jo 



(45) 
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This can be simplified further using relation (|37|), which in the chosen gauge at T = 
becomes 

A,(0) / dx [q{x) g2 + g^ (g(x))] = 2 dy P(l, y) y . (46) 

JO Jo 

so that (|45|) takes the form: 

u = A,(0) / dx q{x) g2 (g(x)) - 2 / dy y P(l, y) = -A,(0) / dx g^ {q{x)) . (47) 
Jq Jo Jo 

For the 3-SAT problem this reads: 

« = --^A,(0) f'dx {l + qix)f . (48) 



128 'Jo 

We conclude this section showing in Figure @ the probability distribution P{x, y) 
of frozen fields at T = for different time scales t^. From the figure it is evident that the 
distribution of the field y varies continuously from a Gaussian, for very short time-scales, 
to a double peak distribution, for the longest time-scales. 



P(x,y) 




Figure 4. The local field probability distribution P(x, y) for the 3-SAT at T=0 in the 
Sommers' gauge. 



6. Thermodynamics of the highly constrained 3-SAT problem 

In Figure (^ we show the entropy density as a function of temperature down to 
T = 0. For each temperature, including T = 0, the data are obtained using the gauge 
appropriate for that temperature. For comparison, the entropy computed within the 
Replica Symmetric, 1-RSB and 2-RSB solutions [Q are also plotted. As it can be seen 
from the log-lin plot, the 2-RSB solution is a very good approximation but yet it is 
inexact when T < 0.016. The entropy is zero for T = as confirming the conjecture of 
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Ref. for the behaviour in the UNSAT-phase. As expected, s vanishes quadratically 
with the temperature. 




I I I I I I I I I I 

0.025 0.05 0.075 0.1 0.125 

T 



Figure 5. Entropy density of the 3-SAT model for large number of clauses. The 
Replica Symmetric, 1-RSB, 2-RSB and oo-RSB solutions are plotted. For the latter 
s(0) — 0. On this scale the 2-RSB and oo-RSB are almost indistinguishable. 




T 

Figure 6. Same as Fig. (|^) The improvement got in the low temperature region by 
breaking the replica symmetry is clearly seen. The data for oo-RSB are reported as 
circle to distinguish them from the quadratic fit. 



Finally in Figure (J^) we show the energy density. 
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Figure 7. Energy density for the 3-SAT problem at high connectivity (a 3> etc) in 
the 2-RSB and cx)-RSB solutions. The coincidence between the 2-RSB energy and the 
cx)-RSB energy is valid up to order 10^'*. 



The quantity plotted is actually u(T) — y/a/iS, where a is very large. Equating 
the internal energy to zero we can determine an upper bound for the critical value Oc of 
the ratio of the number of clauses to the number of variables that marks the transition 
between the UNSAT-phase (in which we derived our asymptotic model) and the SAT- 
phase, where the energy is, by definition, always zero, at T = 0. Using the oo-RSB 
solution we get o;^-^- = 7.10969. For the 2-RSB solution it was already a^-^- = 7.11400 







7. Conclusions 

We performed the study of the Replica Symmetry Breaking solutions of the 3-SAT 
problem in the limit of many clauses, mapping it in a poorly diluted spin glass model 
with long-range random quenched interactions. The mapping to a statistical mechanics 
model was carried out introducing an artificial temperature and taking, in the end, 
the limit T — ^ 0, to recover the original model. We found that the structure of the 
solutions to the problem is of the oo-RSB kind: in order to get a stable solution the 
replica symmetry has to be broken in a continuous way, similarly to the SK model 



(in external magnetic field). The oo-RSB structure holds down to the interesting limit 
of zero temperature. 

No phase transition is expected in the UNSAT phase, other than the SAT-UNSAT 
transition occurring at a = etc — 4.2 Therefore we expect the same oo-RSB structure of 
solutions of found for the over-constrained case to hold also in the critical region. 

From the value of the energy at zero temperature we find the upper bound 
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< 7.10969. to the critical value of the number of clauses per variable. Even if 
this is of the same order of magnitude of etc — 4.2 yielded by direct numerical 
simulations, it is still too large. We recall that such a value has been obtained through 
a first order expansion in Ij^fa. In order to get a better approximant other terms 
should be considered, possibly more then one since we are dealing with an asymptotic 
expansion and therefore nothing guarantees that the second order corrections are small 
and in the right direction. 

Finally, as by product, in the present paper we worked out a precise procedure to 
get the oo-RSB solution of a general class of models that, besides the over-constrained 
3-SAT model, include SK, p-spin and, more generally, models with any combination of p 
interacting terms. We presented the solution exploiting a variational method, introduced 
by Sommers and Dupont , which has the advantage of being easily implemented on 
a computer for any temperature including T = 0. As a consequence the numerical code 
developed to solve the present model can be applied to the whole class of models without 
any relevant change, providing an efficient tool for the analysis of the structure of the 
solutions of a large number of spin models interacting via quenched random couplings. 
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